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ABSTRACT 

This paper reports an expeaiment designed to test 
theoretical relations among fast problem solving, more complex and 
slower problem solving, and research concerning fundamental memory 
processes* Using a cathode ray tube, subjects were presented with 
propositions of the form is in list X" which they memorized. In 
later testing they were asked to decide whether simple and compound 
statements involving the memorized facts were true or false, "^Solfltion 
times for the test problems were measured. Contrary to hypothesis it 
was found that the compound problems of the form "Y is in list X and 
Z is in list X"" were solved as rapidly as simple statements, A 
modeling technique is ^^uggested and rationalized. Suggestions for 
further research are offered, (SDX 
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Ab^t'5§ct 

A class of simple .problem solving tasks requiring fast accurate 
solutions Is Introduced. In an experiment subjects memorised h mapping rule 
represented by lists of words labelled^ by cue. words and made true/false 
decisions about conjunctions of propositions of the form, '^Y Is in the list 
labelled by X," written •'X-->-Y'\ Response times are analysed using a 
••stage modellng^^ technique where problem * solving algorithms are con^qsed 
using a small ^ set of psychological operations that have real time 
characteristics' specified parametrlcally. The theoretical analysis shows 
that response time performance is adequately described in terms of the 
sequential application of elementary psychological operations. Unexpectedly 
it was found that the proposition •'X*>— Y and X->-Z^* was verified as 
quickly as the apparently simpler •'X->-¥*'. A case is presented for the 
modeling technique as applied to memory and problem solving tasks In terms 
of theoretical parsimony, statist leal simplicity, and flexibility in. 
Investigative empirical research. Suggestions, are made as to possible 
theoretical relations among fa^,t problem solving, more con^lex and slower 
problem solving, and research in fundamentlEt]^>memory processes* 
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Introduction 



The dotninant theoretical approach to the analysis of problem solving 
has been to construct a formal model, often in t|ie form of a computer 
program, that simulates some qualitative aspects of human problem solving' 
performance such as the protocol sequences observed in deriving logic 
theorems (Newell & Simon, 1972). In these analyses eiiq>ha8is is placed on 
the integration of elementary information operations into a problem solving 
algorithm while less attention is given to the elementary operations 
themselves* An approach that has been relatively less well explored is to 
specify the processing time implications of proposed algorithms and. to 
determine whether observed hinnan response times (RT's) are consistent with 
the predicted pattern* From a statistical point of view, problems that 
require several minutes to solve or involve extensive searching for a 
solution (e.g., looking for the best move in a chess position, de Groot, 
1965) might be. expected to have large RT variances even for an individual 
subject such that it becomes inq)ractic^l to model the fine details of RT* 
However,^ for sickle problems where human subjects are easily able to respond 



correctly In a matter of a few seconds » it should be possible to verify the 
processing time predictions of specific probl^ solving algorithms. 

One method for deriving RT predictions is to describe problem 
solving algorithms in tei^ns of the sequential application of a set of basic 
psychological operations (procedures » subroutines » or "stages'O each of 
which requires real processing time and has some probability of producing an 
error. Leaving the details for later discussion^ the theoretical RT^ for an 
algorithm applied to a particular problem can be described as the sum of the 
processing tijoes of the operations applied and the error rate is roughXy 
1 minus the product of the correct probabilities of these operations. ^ 
alternative technique for making RT predictions is to assign computational 
complexity measures to tl^e basic operations and to derive the complexity of 

an algoi^thm as the sum ^of the complexities of its component operations; 

linearly related to 

computational complexity is then directly interpreted as/ theoretical mean 
RT. This complexity assignment method yields the same description of mean 
RT*s as does the corresponding stage model although <lt does not describe 
higher RT moments. "Mote that both methods are easily generalized 

to take account of the possibility of mixed (randomized) strategies for 
applying available algorithms. 

4 

On a general theoretical, level» the RT analysis of fast accurate 
problem solving can be a valuable source of evidence in deciding on a set of 
basic psychological operations used in human problem solving. The 'case is 
similar to that for chronometric studies of linguistic comprehension (Chase 
& Clatk, 1972) where alternative representations of propositions can 
sometimes be discriminated by constructing RT models for processing 




propositions to make true/false decisions. For problem solving theories it 
is desirable to build algorithms working with a set of elementary operations - 
which have some preferred characteristics, such as corresponding to 
procedures or subroutines that can be conveniently written as logical units 
when programming in a particular language, or being general in the sense 
that the same set of operations can be used in solving several types of 
problems. Another preferred characteristic is that the set of operations 
has "psychological validity" insofar as real time prpcessing aspects of the 
operations can be defined and verified in observed RT performance. 
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Algebra step problems^ 



.To pursue these ideas m experimental task was sought where subjects 
would learn a set of rules (e.g., the moves of pieces in a board game, or a 
mapping of one set of objects into another) and be required to solve 

i 

true/false problems by repeated application of these rules. It was thought 
that a model for the single application of a rule could then be extended to 
a model for the entire problem solving task by specifyln^the way rtiles w^e 
applied to solve a problem. 

Consider a small finite seh X and a rule that assigns to each 
element of X a subset of X. Such a rule can be written in the form of a 
transition table such as that in Figure 1 which was used in an experiment to 
be described later. 

Figure 1 about here 

A memorised transition table, say where X is a set of conscWnt-vowel- 
consonant (CVC) words, might be represented aaj^lists" in some memory store 
with "addresses" corresponding to the /etfimfiu^ts oy X. One of the mcrat basic 




propositions that can be made^about a particular transition table isv that x^ 

is mapped into a list that contains^ x^^, written Xj as a mapping 

diagram, where x-^ and xj are variables standing for elements of X; this 

proposition is either true or false. A subject who has memorised a 

* 

transition table can be presented with the proposition Xj^^— Xj and fee 
required to make a true/false decision iising his knowledge of the rule as 
defined by the table. In the experiment to be described, subjects were 
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Figure 1 - ASP Transit-ion Table 
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presented with logical "and" conjunctions of these simple propositions and 
RT's for a tnie/false decision were measured. The propositional forms or 
problem types used are listed in Figure 2 in three groups (A, B and C) 
according to the geometric shaped of th^,.mapping diagrams. 



Figure 2 about iiere 




A problem is true if and only if all thfc^-prtJptfsitions^j^^ 

arrows or links are true; if just one link is false then the problem is 

false. For example, P( .) in Figure 2 is true only if Xi;-^x^ and 

Xj-v-x^ and Xj^-V-Xj^ ; it is false if any one of these propositions is 

false. Similarly P(-<) is true only if x-^->-x^ . and x^->-x^ and 

^jr*^^X* P(>^Oon^^f X;~>-Xj^ an<i^ xj->-X|^^ and ^^^-^xj^, 

In the experiment subjects memoriaed transition tables o?*^he form 

represented in Figure 1 where the elements of X were CVC words, and were 

tested with problems of the sort illustrated in Figure 2, Representing a 

transition table in memory as stored lists, an Individual link, ;Cj , 

could be verified true or false by using the cue x,, to "access" the 

appropriate list in memory and then "scanning" the probe xj against this 

list for a "match"; if a match is obtained then the link is true and 

otherwise false, A model for the verification of the conjunctive 

propositions could then be obtained on. the assumption that verification 

proceeds one link at a time in some specified order. These notions are 

developed in the discussion section below. Since it is possible to verify 

mapping ^diagrams by checking each link J^n a step by step manner, the test 

items used in this task are referred to as algebra step problems (ASP) . 
for each problem type 
ASP items /were selected from a computer generated listing of all 



5 





possible Items given the transition table (Figure 1)\uch that within each 
problem type there were an equal number of true and false Items. For false 
problems exactly 6ne link was false, and for each type the false link 
'occurred with equal frequency at each link position. In addition an effort 
was made to match the frequencies of occurrence' of CVC words between true 
and false items within each problem type so as to avoid a poi^^ble source of 
response bias. The total pool of about 400 distinct items was divided into 
four blocks; the same P(**) items obcuring in each block but otherwise there 
was no overlap. Denoting blockig by , B^^, Bj, B^, subjects- were tested 
oyer six sessions with one block ^r day^in the order B^ B^^B^Bl^B, B^, where 
blocks and trials within blocks were randomised for each subject 
Individually. A set of nine CVC words was randomly assigned to the abstract 
transition table scheme, for each subject: no two subjects had identical 
transition tables although all tables had the same formal structure. 

The e3q)erlxtent was run using an Imlac Corporation FDS-*1 cathode tay 
tube (CRT) display and keyboard, interfaced with a FDP-10 con^uter. Six 
female subjects ran for seven sessions; the first session was devoted to a 
transition table learning drill (subjects did not memorise thei^^ transition 
^^S^tSB^xlox to th& first session), and the remaining six sessions were used 
for ASP test items. On a single trial of the drill a cue word was presented 
on the CRT apd the subject was required to type tjje- appropriate list in 
serial order (since the CVC words used had unique initial consonants, the 
subject typed only the first letter of each word an4 the computer completed 
the words with suitable horizontal spacing) . On completing her response to 
a cue the subject pressed the keyboard opacebar and the correct list was 
printed horizontally directly beneath the typed response, providing feedback 
and an opportunity for study. Permutations of the nine cue words were run 
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and following each permutation the subject was told her percentage of 
correct responses and the time taken to respond to all the 'cues. Subjects 
were required to participate In the drill until they could consistently 
achieve perfect accuracy with a response time under 25 seconds; all subjects 
met this criterion within 30 to 50 minutes of the drill. 

In the problem sessions ASP Items were displayed at the center of 
the CRT and subjects responded true /false using two keys tm the lower row of 

Y 

the keyboard. The subject Initiated trials by pressing the spacebar 
following a ready signal. Items were preceded by a 1 second duration 
fixation cross and appeared just ta the right »of "the cross, remalnj^ on the 
screen until the subject responded. Immediately after respojralng subjects 
received a feedback message indicating correct/error and response time. 

Before the first problem session subjects were shown examples of the 
seven problem types and told to respond "true" if and only if all the links 
in an item were true and to respond "false" as soon as they knew that one 
link was false. Subjects were Informed that there were an equal number of 
true and false items within each problem type on each day and that false 
items had exactly one link which was equally likely to occur in any 
position. On the first day of problems subjects were instructed^ to be 
coiq)letely accurate for the initial thirty or forty trials and then to 
increase their speed as they got a feeling for the task. For subsequent 
testing sessions subjects were instructed to respond as qiltckly they could 
without making more than about one error in twenty trials on average. ^ 
Subjects were explicitly instructed never to guess and never to "think 
twice" alJout their response once they had made a decision. 
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To • eliminate early practice effects ^nd-' to facilitate the 

/ / 

observation of stable task strategies the data ^or. each subject from the 

/ 

first of the six testing sessions was discarded together with the first ten 
trials of the remaining five sessions, yielding on the order of 550 trials 
per subject. Only correct RT's excluding outliers were analysed. Correct 
RT histograms were plotted separately for each probiejD type, for both true 
and false responses, and for each siabject to identify possible outliers. 
Response times falling more than 1 second above the main distribution as 
determined by the mode and the coptigiwus talla were, eliminated; such 

outliers constituted about 2Z of the correct RT data. 

> 

^ — " — - — 

Table 1 about here 

Due to the coiiq>lex description of the ASF items it is not possible 
to represent all aspects of the data simultaneously in a single graph or 
table. However, by collapsing across various subsets of the data we can 
obtain a reasonable picture of majdr effects^ ^ich can then direct more 
detailed modeling and statistical evaltiation. Since plots of the data for 
individual subjects showed subjects to be qualitatively coiiq>arable, the RT 
data for aU^ six subjects Was pooled to simplify the presentation of 
results. Table 1 presents RT and error rate data classified by problem type 

and position of the false link (if any) . The notation P( ) TTF 

Indicates that the third link from the left waa false} P(>-) PTT that the 
upper link of the branch (>) was false; . P(-<) TTP thtft the lower link of 



Table 1 - Group RT Means and irrors 

problem type and position of false 



n 



link) 



type 


false 
•link 


obs 
mean 
(msec) 


th 

mean 
(msec) 


obs 
s.d. 

(msec) 


error 
Z 


'total 
N 


P(-) 


T 


1576 


'>529 


590 
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^-die branch (<) was false* Observed means and variances and theoretical 
means (derived from a statistical model introduced below) are averages 
across subjects weighted by the numbers of correct RT's observed. 



Figure 3 about here 



Jigures 3 and 4 are based on the^-data of Table 1; curves represent 
theoretical mean RT. Figure 3 plots true and false mean RT by problem 
types. A striking feature about these data are the 'following approximate 
equalities of mean RT's obtaining among the problem types; 

P(<) - P(-) and P(-<) « P(~) 
P(>) . p(_) and P(>-) - P( ) 

Of course these equalities hold among averages including quite distinct 
items within each problem type, but thdy do suggest that the time to verify 
a ttft branch configuration (<) is not substantially different from the time 
fot a simple link (-) . tn contrast, verifying two links in the (>)* 
configuration appears to take the same time as two links in the ( — ) 
configuration. In what follows the (<) configuration will be referred to as 
^ <^Q^^3.''B probe link and (-) as a single probe link. 

Within each of the problem groups RT increases with the number of 
links. If a sequential processing of links is assumed then the slopes of 
the true curves directly reflect the average time taken to verify that a 
link is true. Note that the three true curves, plotted in Figure 3 have 
approximately the same slopes, which together with the eqtialltles remarked 
above is consistent with a sequential processing account. A way to 
investigate order in sequential processing is to examine false RT's for each 

« 
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problem type In a group as a function of the position of the false link 
aa^usditg. that subjects responded "false"' as soon as they discovered a false 
iink» Figure 4 illxistrates graphically this order of processing analysis. 



Figure 4 about here 



Figure 4a shows that for group A problems Kt Increases as the false liijk is 

■ t 

moved from the first to the third position with a slope about the same as 
the true slopes in Figure 3: this Indicates a strict left/right processing 
order. FJ^re 4b shows that for P(-<) the tail link (-) is almost always 
verified before the left branch (<), while within the branch there is no 
3trong up/down processing order. This is interpreted as consistent with the 
proposal that the double probe link is verified in one step (i.e., not as 
separate simple links) which ic^lies that there ^should be no~ up/down 
processing order as such» Figure 4c presents ^ more complicated story for 
group C. While link processing for this group tends to be up /down on the 
right branch (>) and branch (>) before' tail (-) (i.e., left/right) in P(>-) 
this order cannot be strict since the* RT slopes as the false link 
position moves are noticeably less than, the true slopes in Figure 3. A 
probabilistic order of processing is appropriate for group C problems. 

Figure 5 about here 0 



The verification of a link is in some respects similar to memory 
scanning tasks (Sternberg, 1969a) that require subjects to decide whether a 
probe symbol is contained in a memorised set of S3nnbdls. For an ASP 
transition table the number of elements in a list labelled by a cue word is 
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Cue Set Size 



5 - Mean correct BT's for problem type p(-) 
plotted by cue set size .and true/false. 



referred to as the cue set size; Figure 5 plots true and false RT's for P(-) 
by cue set size to illustrate set size effects analogous to those found in 
memory scanning tasks. The true and false curves are separated by a 
constant, suggesting a simple additive effect on RT of the process 
differences between true and false link verifications. 

Errors were infrequent under the speed/accuracy instructions given 
the subjects; the error rate over all conditions and subjects was 5.2 
percent • Group error rates broken down by problem type and position of the 
false link are presented in Tables 1 . While the authors recognise the 
possil^ility of important theoretical relations between response times and 
error rates as for example suggested by Pachella (1974) among others^ a 
rigorous analysis- relating the two was not performed for the data presented 

here. This omission is partly justified by the empirical observation that 

< 

while mean RT's showed a consistent pattern across subjects, error rates did 

not. Also, from purely statistical considerations when data ±s so finely 

classified that some , classifications have twenty 'or fewer observations, 

error rates may not be sufficiently reliable for the analysis of data from 

an individual subject^ whereas RT's may stflll be meaningful in providing 

insight into psychological processes. 

s 
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Theoretical Analysis 

f 



Suppose that for a partflcular ASP Item we have beep glv^ a 

description of the sequence of psychol6glcal operations used to solve It* 

The stdge modeling technique to be used here assigns to each operation or 
stage, S y of the processing a tuple of parameters, 

< M(S), 6(S) > 

corresponding to the theoretical mean and variance of processing time 
associated vd.th that stage. In cases more general than that considered here 
this tuple may become a family of tuples corresponding to var|Lous states of 
the cognitive system that could exist when the stage opera^s (l«e., stages 
are specified conditionally) or tuples may contain dldltlonal parameters 
stich as higher RT moments or the probability of a processing failure In that 
stage. If stages S, , S^,..., S^^ are applied In sequence to process the 
item then the RT mean and variance for the dtem are slmily, ^ 



.(RT) « £/*(\) and /(RT) - Z tfk 



) 



4»» 4=» 

The . addltlvlty of ^variances follows from the assumption that stage 
processing times are stochastically Independent. Now suppose that there a^ 
two sequences of , stages that could be applied to the Item, S^, S,^^ 
and S^^ , S^^ » • • • » S^m^ that these two sequences ar6 observed with 

probability p and (1 -p) respectively . Let , 

/^•v = L /^(Sjj ) and v\ = £ 6'(Sij ) 1-1,2 

Then the mean variance of the overall RT are, 
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1 

<5*(ET) - p< + '(1-p)«\ + p(1-p)(;A,->t^)^ 



Without goixig^ Into further: detail, similar expressions can .be derived 
vhenever RT is assumed to arise from the probabilistic mixture of sequences 
of stages .^^ 



Proceding on the basis of the observations made in the results 
section above, a stage model was constructed uslng^ a small number of 
Inclusive stages that are Identifiable (i.e., in the sense of unique 
parameter estimates) and that have direct theoretical Interpretation. These 
stages are; 

stage * stage description 

verification of a single probe link 
with cue set size of n 

verification of a double probe link 
with cue set size of it , 

R orientation, attention, perception 
and miscellaneous set-up and 
bookeeping processes 

D decision and response processes that 
^ differ between "true" and 
"false" responses 

Processes involved In the verification of single and double probe links have 
been summed together in the and parameters respectively. Due to the 
problem of Identifying parameters it is not possible to make definitive 
interpretations of the stages R and D. The R stage includes all those 
operations which are in common across problem items, such as attending to 
the CRT display or executing the motor components of a keypress response; in 
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addition K may be regarded as incorporating incidental processes required 
for the logical completeness of the model such as recording the input and 
output of stage operations. Any processing differences between true and 
false responses, including handedness, are incorporated in the D stage* For 
the experimental data false responses are slower than comparable true 
J responses.; the D parameters reflect this aspect of the data* 

Derivation of theoretical expressions 

^ y 

The derivation of expressions for thedftretical RT means and variances 
will be illustrated by examples sin<^ there is insufficient space for an 
exhaustive treatment* In the following let n^ be the cue set size 
associated with the s3nnbol x-^* 

Example 1: 3C;->--«j (T) 

A 

To solve this simplest problem the subject need only verify one link; hence 
exactly the stages V^^ and K occur* Then, 

- /a(V^.) + /*(K) 




Exanqple 2; x.->— x.-^x^j-V-Xj^^ (TFT) 

Assuming that P( — r-) has a strict left/right processing order the subject 

111 
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first verifies that x^-^-Xj is true and then finds that x^--^x^ is false; 
the subject responds "false'* as soon as she finds this link so that only 
stages V^. , V^. , K and J) occur, ^ 



>.(RT) = A^(V„.) + /-^(V^^) +^/^K) +/^(D) 



Example 3: x.->-x.<^ (TFT) 

It is assumed that the double probe link (<) is verified in a single 
operation, W^^ , and that the tail (-) is checked before the branch (<) , so 
that the stages are , *i,K and D. 

^■(RT) = /a(V^ ) + /a(W^.) + /^(K) + /^(D) 

^■'(RT) « C'CV^.) + ^''(W^,) + <S^(K)-+ d'(D) 



Example 4: ^^^^^f 0^) 




A probabilistic order of processing was suggested for tjrpes P(>) and P(>~). 
This order id.ll be. defined by two probability parameters. Let q be the 
probability that within a right branch (>) the upper link is checked before 
the lower linic, and let r be the probability that for P(>-) the branch (>) 
is checked * before the tail (-) . In Example 4 the parameter r is not 
involved. With probability q the stages are V^^^ , K and D, and with 
probability (1-q) the stages are V^^ , V^. , K and D, with the result th^t, 
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(j'-CRT) - /(V^.) ' (l-q) ^^(V^ ) + <j'^(K) + (D) 

The expression for t5*'(RT) is that for the probability mixture of two 
sequences of stages* • ^ ' 

■ 1- 

.Pxample 5: * ^>k^^->-.Xjj^ (TFT) 

In Example 5, with probability ' rq the 'stage's are V;^. , V„. , K and with 
probajbility r(1-q), V^. , K and D; with probaObility i1-r)q, vl^ , , V^. , 
K and D; and with probability (l-r)(1-q}, , V^, and Hence, 

+ >la(K) + /a(D) ' " 

+ (S'(K) + d-(D) + q(1-q)[ (V^.)]^ 
+ r(l-r)I ^(V^jf 

The expression for (RT) is an algebraic simplification of a general 
expression. ^ ' 
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Exanqple 6: JP^^T^^ (TTT) 



X 

J 



Since all links are tvu^e the same stages mtjst occur ^^atever the order of ^ 
proceeding* Consequently » 
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^(RT) - y^C7^. ) +V(V^.) +/*(V^J +>^(K) 

cj^(Er) o'-(v^.) + <,*(v^.) + d\v^^) + <r(K) 

These exaiiq>les should convey the gist of the statistical model. 
Not:e that for every ASP item th^ 'theoretical RT mean and a variance can-be 
expressed in the following canonical form. 



/*(RT) o a, /a(V,) + a^^(Vj) M- a^XV^) 
+ a^^^(Ky + a,/*.(D) 



» < 

+ a^<^(K) + a,<5*(D) +• b*" 

where the a-^ 's (i»1,...,7) can be Interpreted as the average number of 
times the corresponding stage occurs, and is the "mixture variance" 

(i.e. the variance added by mixing processing strategies ' irfiere strategies 
may require differing amounts of time). Writing the row vectors, 

e = < ;*(V, ) , j^(Vj , jAV^) , /*(WJ , /^(Wg) , j^(K), ^D) > 

g = < <^V, ) , <S>3 ) , 6(.Vj) . 6XVj) , ^(K) , 60) > . 

a - < a, , a;, a,, a^, a^, a^, a, > 
the canonical fonds become, 

/jl(RT) = a e^ , <j''(RT) - a v^ + 
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where e^ is the transpose of e and the transpose of v. For all 

true items and for false Items in groups A and B, each Is an integer 
and b^"0; for false items in group C the a-^^'s may be functions of q and 
r, and 1?'>0 is a function of e , q and r. Note that ASP items can be 
classified according td their coefficient vectors, a , and mixing variances; 
b^; under the model this classification is a full specification of the 
items. For the items used in the experiment forty-six such classification 
categories occurred* 



Statistical Evaluation 

A discussion of parameter estimation and statistical techniques is 
presented in the appendix. Best estimates of parameters were obtained for 
each subject by numerical methods using a quadratic loss function, and the 
fit of the model to the RT data was primarily evaluated by constructing 
mlnlminn simultaneous confidence regions containing all the RT means and 
variances predicted by the modeJL. Parameter estlmtes are given in Tables 2 
and 3; statistics are listed in Tables A and 5. 



Tables 2 and 3 about here 



For the mean RT data the statistics in Table 4a show that while the 
model does account for a substantial percentage of the between and total 
variances (PBV and PTV columns), the maximum modulus t test applied to the 
group suggests that the model is probably not a complete account of the data 
for every subject in the experiment (g* for the group is .004 which is the 
probability of observing a t* value of 4.44 or greater). 
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Table 2 - Parameter Values, Averaged. Estimates 

, (msec) 
single probe (-) verification: 

cue set size 1 VI 676 

. cue set size 2 V2 1169 

cue set size 3 V3 1146 



^"tipuble probe (<) Verification: 



cue s.et size 2 W2 924 

{ 

^ ^ cue set size 3 W3 1206 

s^t-ijp processes K 446 

true/false difference ' D 466 

probability of up before down on (>), q « 0.85 

probability of (>) before (-) on (>-), r « 0.89 
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Table 3 - Paraaeter Values^ Individual Subjects 



Subject 1 Subject 2 ' Subject 3 



• 














V 1 


o/o 




i 




571 


0 


V2 


1545 


676 


867 


293 


. .1001 


450 


V3 


15J7 


720 


913 


593 


775 


446 




91-8. 


575 


789 


399 


748 


398 


W3 


1144 


442 


986 


421 


768 


284 


K 


475 


1 


545 


'l - 


394 


0 


D 


729 


1 


346 


489 


■452 


0 


q 


0.79 




, 1.00 




0.7a 


r 


0.88 




0.48 




1.00 






— Subject 4 


Subject 5 


Subject 6 


- 


M 




>*• 






<S 


VI 


956 


0 


594 


259 


607 


.532 


V2 


1375 


671 


1430 


635 


798 


230 


V3 


1348 


729 


127.1 


646 


980 


636 


W2 


1156 


689 


1045 


3.1 


838 


78 


W3 


1582 


720 


1455 


485 


1301 


88 


K 


292 


;1 


462 


307 


506. 


414 


D 


459 


1 


302 


2 , 


506 


1 


q 


0.61 




0.92 




1.00 




r 


1.00 




0.99 




1.00 






* 
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Table 4 about here 

In Table 4a two of six subjects have g* > .10 Indicating a good fit of the 
model for these individual .data, and in Table 4b g* > .10 for three 
subjects. The third column of Table 4 gives the number of points lying 
outside a «90 simultaneot2S confidence interval; any such point implies that, 
^g* < .10 . 

Table 3 about here 



Table 5 presents statistics for RT variances. Due to large sampling 
variability of variance estimates, the g* statistics are not very 
informative since many models would be acceptable within wide limits of 
variability. The third column of Table 5 compares the model to the 
hypothesis that all RT variances yp^^^e same, in terms of the proportion of 
points for which the model makes a more accurate prediction. Evaluated 
using this statistic the model does no better than 4:he^ "same" hypothesis 
alUiough both ^re acceptable given the variance of estimators. Since the 
' averaged RT variance parameters presented in Table 2 appear to . be orderly 
they will be discussed although no strong conclusions should be drawn. 
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Table 4 - Statistical Analysis for RT Means 



points 

subject t* * g* outside PBV PTV * 

90Z 
region 



if 

A. model classification (46 points) '^^ 



1 


4.44 


0.000 


2 


79.7 


42.7 


2 


2.78 


0.182 


0* 


76.7 


35.6 


3 


3.32 


0.029 


2 


78.6 


40.7 


4 


4.44 


0.000 


1 


79.3 


36.5 


5 




0.007 


2- 


57.7- 


28.1 


6 . 


2.84 


0.157 


0* 


68.7 


/27.8 


groiq> 


4.44 


0.004 


3 


73.5 


35.2' 



B. type X false link classification (23 points) 



1 


3.28 


0.014 


1 


85.6 


41.5 


2 


1.74 


0.697 


d* 


84.3 


32.1 


3 


2.42 


' 0.198 


0* 


89.5 


37^^ 


4 


3.01 


0.036 




'^83.7 


35.8 


5 


3.44 


0.008 


1 


64.5 


26.6 


6 


2.67 


0.101 


0* 


71.0 


24.5 


group 


3.44 


0.049 


2 


79.8 


33.0 



33 



Table 5 - Statistical 'i^lysis for RT Variances 
(model classification, 46 points) 

proportion 
subject t* g* - model 









vs 
"same 




0.83 


0.999 


. C.478 


2 


0.63 


0.999 


0.500 


3 


1.00 


. 0.999 


0.348 


4 ^ 


t.07 


0.999 


0.522 


5 


0.85 


0.999 


0.478 


6 


0.56 


0.999 


a. 391 
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Discussion ' ^ 

I of < 

Stage modeling has been con ceived /in terms of a formal processing 
langiiage description of memory operations: stages are analogous to 
procedures or subroutines; perhaps probabilistic in their execution, 
organised by' call sequences into memory processes. Within such a stage 
modeling framework various levels of detailed description are possible. For 
example, one might consider macro stages such as "perception", '^memory" and 
"response", or comparatively micro stages such as "input the symbol in 
position p of the stimulus array" or "compare the code for symbol X with the 
code for symbol Y". No particular level of detail can * be regarded as 
preferred: theoretical descriptions in stage terms must be evaluated with 
respect to the relevant data. Hovrever, the stage modeling framework ^oes in 
principle relate all 'levels of description in terms of the nesting of 
procedures in call sequences, thus providing the possibility of consistently 
treating the results of sinqple and relatively more qomplex laboratory tasks 
with the same overall processing model (Atkinson & Wescourt, in press) . 

A stage mpdel can be most productively regarded as a rational basis 
for the construction of statistical models* Each statistical model stemming 
from a stage model can be evaliiated with respect to the data, sucMsses and 
failures yielding new information about the data possibly not apparent on 
inspection or available from other analyses. In general it is not necessary 
that every statistical model derived from a particular processing language 
description be "succesisful", but only that some are and that these provide a 
useful charitcterisatlon of the data. Of course, if a stage model were taken 
as a literal model of a specific real time process, say specific 
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interactions among brain centers and layers of brain tissue, it would be 
important to verify all the statistical models derived from the stages 
theory. However^ fdr the analysis of cognitive performance the authors 
regard stage models as non-literal information processing descriptions from 
which statistical analyses are derived that provoke a deeper and more 
adequate characterization of patterns present in the data. 

The statistical models for RT means and variances developed above 
may be regarded aa an intermediate level of stages analysis appropriate to 
the level of observable data: it does not explicitly describe either the 
coiiq>onent processes of individual link verifications or the overall control 
structure in which the problem solving algorithms are embedded. Since these 
additional levels of analysis are of theoretical interest, the discussion 
will tukn to bridging these conceptual gaps. The following stages analysis 
of single and double probe link verification is given; 
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determine whether single or dpuble ■ 
probe (s d) .1 


- 




input cue m 






£LCcess memory list associated with cue « ■ 






input probe 1; if d then input probe 2 ' ■ 






reset match register 1; if d then reset 
Xeglster^ 2 

unpack an element from the memory list ^ '| 








k 


match the element against probe 1 and 

increment match register 1 by the value 1 
of the "goodness-of-match"; ■ 
if d then match against probe 2 and ^^cts^ 

Increment register 2 m 






if the entire list has been unpacked' then 
continue else. return to 


* 




if d then add match register 2 to register 1 # B 






if s then if the value of match register ^ 1 

exceeds a criterion c^ then return tru^ ■ 
else return false; m 
if d then if the value exceeds c^ return - 

true else return false « 




Note that the analysis Is essentially an "exhaustive scan" nodeli where 




matching Is not necessarily all-or-none> and where the representation of 1 




lists In memory and the co«>ordlnate retrieval or unpacking process may be 




more Involved than reading from a list of symbols at a uniform rate* I 




Representing a list as a. cluster of symbols bound to a memory node by _ 




associative linkages and defining retrieval processes in terms of this B 


: 


representation would be one way of conceiving of an unpacking operation with . m 

1 




more cosqplex characteristics » although such %u±lt In" characteristics may ' 




have limited conceptual and theoretical Interest* ■ 




The claim Is that this model of link verification Is consistent with 
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•the stage parameter estimates in Table 2; for the sake of simplicity only 
- the average values of parameter estimates are discussed. The parameter 

\ • 

values In Table 2 may be qualitatively summarised as folloWB; 
>*(V,) < jj.(Vj ' ^(Vj) 

d*(v,) < ^(yj < <s*<V3) 

t 

This summary can be regarded as an hypothesis that, within the sampling 
variability of the parameter estimates, is not disconf irmed. A problematic 
aspect of this summary is that /^(M^) < XVj^) by 245 milliseconds,, yet 
/^(Wj) « /^(V^). This result may be attributable In some way to the fact 
that for Wjj^ the number of probes is the same as the* cue set size, but in the 
absence of additional controls no ad hoc explanations are offered. 

If it is assumed that stages L^^^, and account for the major 
part of link verification time, then a gross slmllilarityN^ould be predicte'd 
between single and double probe links. With suitably complex 
representations of lists the mean unpacking time for lists of lengths 2 and 
3 may be comparable, yielding /a(V^> « /^(^j); the speed of Vj could be 
explained by the simplicity of the representation for a list with a single 
symbol requiring fewer unpacking* manipulations. 

0 




single and double probe link verlflcatj^ons differ In stages and 
L^. If the matching process Is probabilistic (e.g., due to variable 
li&perfect coding of symbols) then the final tnatch value In register 1 will 
be distributed differently for single and double probes (e.g., double probes 
will have greater mean and variance for both true and false links). This 
together with the two criteria, c^ ani c^, mffeht accotmt for decision 
coiiq>^nent differences In ways similar to signal detection models ^hat relate 
RT to criteria placements in relation to signal ^and noise distributions 
(Thomas, 1971). Itie observed <5^(W^) < arid 6'(}l^) < <t(V^) are 

interpreted as due to such differential effects in stage L^. 

The value /^(D) « 466 is greater than would be expected on the basis 
of handedness aloil^, sugge^ing genuine decision component differences ; 
again this is interpreted as a stage effect.. Since successive IdLnk 
verifj^tlons are require^.^ some ASP items, in order to achieve an 
acceptable error rate/^subjects were instructed to be accurate) it is 
necessary to make a more accurate decision for each intermediate 
verification than would be needed if only a single link were verified on 
each trial. Also since over all items there are morej:rue than false links, ^ 
stage L might be 'Ituned" for a true verification. The demand for increased 
accuracy together with a true verfication expectancy could account for the 
observed value of /^(D). The*|fi>parent constancy ofc /^(D) over pfobleid 
types, even those where only one link is verified, is consistent with the 
theoretical concep*tion that the jsame link verification mechanisms are used 
for all problems without modification according to prdblem type* From these 
considerations it woxild be predicted that encouraging speed over accuracy, 
using only single link problems, reducing the variety of ASP items used 
within a single experiment, **br using multi-link items with more than one 
false link would all have an effect in reducing the value of /a(D) . 
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As an aside, it may be possible to use an eiiq>lrlcal speed/accuracy 
tradeoff to further Investigate the verification mechanisms found In the ASP 
task. A direct implication of the theory discussed above is that under 
speed instructions each link verification will be less accurate as' 
processing is modi^ieff'^or spfijfiAi or cut short, with the results that errors 
will tend to^-lficrease^p^latively more for items with many links coiiq)ared to 
those wljh^^few, an^ that error RT*s for multi-link true items will decrease 
rel^tj^e to coipct RT*s while 6rror RT's for multi-link false items will 
increase. Othek quite different effects of speed instructions might be to 
induce subjects to implement ^faster problem solving algorithms, say with 
some sort of simultaneous verification of links, to "prlme*^ access to 
certain algorithms and retrieval mechanisms in anticipation of the next 
problem, or to adopt sophisticated guessing strategies. The Issues with 
regard to speed/accuracy effects in ASP problem solving are manifold and may 
perhaps be most productively approached by comparing results across 
experiments to determine what effects might be present. 

In stage terms a stable strategy is a problem solving algorithm that 
iB'f WTrm a^ j L fl ed with use. Empirically, stable strategies would be expected 
for practiced subjects who have in some sense developed optimal tasK 
techniques, with the required amount of practice depending on the partlctdar 
task. The present experiment was designed to observe only asymptotic 
performanVie, making it in principle possible to specify a single set of 
algorithms or strategies governing the processing of ASP items. A theory as 
to how these strategies are set up with practice is not developed here; 
however, the authors do conceptualise an Interactive feedback system where 
the state space of the system consists of algorithms and the effects of 
control Inputs are to rebuild algorithms. It is proposed that for tasks 
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Vhfete alternative proceeelng etrategiea are a genuine^ theoretical 
possibility. It may be more appropriate to analyse data ficr trials early In 
the experiment, in tetms of a mixttu:]^ of strategies rather than a single 
Stable strategy/ For -the sake of completeness of th^retlcal conception it 
is asstmied that the strategies for the vatlous problem types are called ^y, a 
dontrolling stage that on each trial identifies the problem type on the 
basis of its mapping diagram configinration and Calls the co^^reSponding 
problem solving algorithm* ' * *^ ^ 

Additional enq>irical work is required to evaJLuate. these conceptual 
analyses of control and component processes « For example, one line of 
experimentation would be to more thoroughly examine ASP verification 
problems, with maniptilations of the transition table ' and problem types, 

i 

Another line would be to examine ASP probletOs more complex than 
verlf ication„ with the idea that such tasks could rev^l more about the 
construction of strategies, that is about bow^pmponent processes are used 
tfo build 'problem solving algorithms. Alternatively the verification of 

* ■ \ 

isolated sldgle and double probe links could be examined in greater 

' f ' 

experimental detail. All these levels of experimental investigation are 
veil integrated within the stage modeling framework, which is, again to 
emphasise, one of the main theoretical motivations for using such a 
framework as aphasia for data analysis* 

From a theoretical standpoint a close relationship exists between 
link* verification and some memory-scanning tasks* In 'both cases a probe 
item i^uat In some sense be coi^ared against a list of symbols in memory to 
determdbce if the probe Is a member of the list* ^A point of interest is 
pKhethar memory-scanning mechanisms that have been investigated in the 
laborat^r can be identified as coiq)onents of relatively more cotaplex tasks 



such as solving ASP verification problems. The model constructed for the 
ASP problems Investigated in this paper can be regarded as an attempt to 
tackle this isstie* About the simplest relation that could obtain between 
memory * scann in g and ASP problem solving would be that the scanning 
mechanisms engaged by stategies to yield intermediate results have the same 
characteristics as those observed with simple memoj^r scanning tasks. Yet 
this need not be so. It is conceivable that as strategies for the more 
complex storage, retrieval and decision making required by ASP problems are 
constructed in the memory system (Atkinson & Wescourt, 197^) new demands 
for rapid access to a larger volume of stored information, for the recording 
of intermediate results which direct further processing, and for controiling 
error rates when intermediate results are combined or cascade in a final 
decision, demand scanning mechanisms having different characteristics. The 
data from the present experiment' are not in themselves conclusive, but the 
parameter values of Table 2 as discussed above suggest that the inferred 
scaxming (link verification) mechanisms and decision processes yield 
values of RT parameters that differ from those typically found in the memory 
scanning literature, There/fs the unexpected result ^£at verifying a double 
probe link is as fast as verifying a single probe link; the fact that for 
single probe links verification tijnes for cue set sizes two and three do not 
differ from each other but are dramatically different from the verification 
time for cue set size one; and the unusually large constant difference 
between true. and false RT*s, Each of these effects is of course subject to 
further investigation and taken one at a time are not without some parallel 
in the memory literature, but the authors bl6lieve that they provoke an 
examination of the issue of how memory scanning mechanisms relate to the 
larger human memory system* It is fair to say that proportionally more 



effort has been devoted to uaravelling the effects of experiioental 
manipulations on basic memory scanning tasks and constructing sophisticated 
and interesting models for these data (e.g. Theios, 1972; Anderson, 1973; 
^evell & Atkinson, 1974), than has beea devoted to Aym^TTfTf g the possible 
roles of memory scanning mechanisms in human memory systems that are 
sufficient to support more involved cognitive processing. 

The stage model developed- for the experiment described here 
characterised each stage by two parameters, the mean and variance of 
processing time; as remarked above this type of model can be generalised to 
Include more parameters such as the probability of an error in that stage or 
higher moments of the processing time distribution. Without changing the 
nature of the npdeling technique, stage parameters could be expressed 
conditionally on the state of processing, as for exaiiq)le on the input to the 
stage from^4>reviQiusly operating stages. Even with these generalisations 
parameter estimatibn and statistical procedures can be derived in a 
o^thematically simple way. Granted that it is one opinion, the authors feel 
that statistical methods such as those described in ttiis paper that are 
based on a formal but flexible model of psychological processing should in 
many cases be both practical and more incisive than the standard linear 
statistical analyses often found in the memory and ^problem solving 
literature. 




.Coa?>ftri8on vltlr Hayes • spy problems 

/ ' ' - 

Hayes (1965,^ 1966) bas reported studies using a problem solving task 
similar to that' of the ASP prbblems defined here. Subjects in Hayes' 
experiments learned a list of "spy" names together with rules about which 
spies could talk to. each other; the list of these "talking connexions" may 
be regarded as a transition table. In the basic experiment, subjects were 
given two spy names and ^««required to find a chain of spy-to-spy 
communications conveying a message from. the one spy to the other. Subjects 
were instructed to "think aloud" and their protocols were analysed with 
respect to the overall time taken to solve a problem, the rate at which 
links in th^e communication chain were generated, and diversions into %lind 
alley" side chains (i.e.,. passing the message to a spy who did not. have the 
connexions to get it to the goal spy). Subjects were able to solve spy 
problems in a matter of a few minutes, occasionally entering side, chains and 
usually achieving a solution chain longer than the Tn-fni™i required^ chain;. 
the reader is referred to the original papers forH^yes' analysis of his 
results. In* terms of the type of theory proposed here for ASP problems, the 
solution of spy problems would be described by algorithms constructed using 
a small set of basic psychological operations and following specjLfic search- 
•and^test methods of chain construction. Insofar as the model stated 
definite algorithms it would have the potential to account for protocols; as 
stage models the algorithms would also make quantitative predictions about 
the pattern of observed RT*s and error rates. • Of course the particular 
theory of ASP problem solving outlined in this paper is not sufficient in 
itself to account f&r Hayefe*s results such as the end-acceleration 
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phenomenon; In addition explicit algorithms woidd have to be constructed and 
demonstrated by coxoputer simulation or by inferential data analysis to 
produce the observed pattern of results* 

The stage modeling technique 



It is worthwhile to eDq>hasi8e the positive aspects of stage modeling 
as a technique for the aiialysis of RT tasks. Interesting arguments related 
to those presented here have been given by Sternberg (Sternberg, 1969b) with 
respect to the so-callfed additive factors method. First as has been noted, 
considering psychological processes as procedures or subroutines in the 
sense of a formal computer language provides an easily conceived unifying 
framework for theoretical analysis and a rationale for investigating memory 
mechanisms as they occur both in simple and conq)lex laboratory tasks. 
Second, from a statisitical standpoint regression models for RT moments can 
be derived from a stages theory in a relatively simple manner, basically by 
counting the occurrences of stages. The parameters in the regression iwdel 
have direct psychological interpretation in terms of real processing time, 
and the parameters can be estimated by commpn analytic or numerical methods 
irrespective of the number of classification categories or the number of 
observations in each category. With regard to predictive power, stage 
models can provide accounts for RT moments of all ordets and together with 
notions of processing variability defined at specific stages can at the same 
time provide an account of errors. Even though the technique is 
mathematically slii5>le, the underlying process representation is that of a 
quite general sequence of random variables (or random vectors) corresponding 
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to the definition^ a discrete stochastic ptOfcesr (vl^ra^afllly of random 
variables with a countable Index sef^-^TTtth very few res trie tjLons (e,g, iflost 




of the random variables csn be asstxm^d to bkNfinite valued). This suggests 
that many inodels of memory procesi<^ will be^*Ni± ieast formally "nearly" 
'equivalent to some stage model ?is defined here. The nature of this 
equivalence can be formalised in terms of the partitioning of the' event 
space of the experiment (i.e^ the set of all possible data points) Induced 
by the inverse mapping of the goodness of fit, measure regarded as a random 
variable. 



Simple and complex tasks 

The algebra step problems introduced in this paper are, like other 
artificial memory and problem solving tasks, not advocated for their 
Intrinsic interest but rather as one experimental paradigm for testing our 
understanding of human memory systems. Fast accurate problem solving has on 
the one hand clear theoretdfcal relations to conceptions of basic memory 
mechanisms and the manner in which these mechanisms come to play in a larger 
memory system, and on the other 'hand it is a bridge to the chronometic 
analysis of more traditional problem solving tasks. While the investigation 
of simple tasks is indispensible it is surely necessary to develop 
theoretical constructions for more complex tasks with equal vigor: the 
chronometric analysis of tasks at the level of ASP problems is intended as 
one step in this direction. In philosophical perspective there is no 
assurance that even a detailed understanding of the models required to 
account for isolated simple memory tasks will automatically lead to an 

r • 
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adequate^ conception of htnaan memory systems that are capable of supporting 
such routine cognitive functitms as the retrieval of ' prepositional 
information (Anderson & BOwer, 1973) or grade school arithmetic problems 
(Suppes, Loftus & Jerman, 1969), The data and analysis presented in this 
paper suggest t^at analysis of RT's on the order of five seconds is feasible 
without undue loss of precision either in the conceptual model or the 
statistical treatment. Across e3q)eriments it should be possible to identify 
the characteristics of memory mechanisms as they occur in merory systems 
where processes involving alternative st;rategies, intermediate processing 
results and^eclsions about subsequent processing , and rapid access to large 
amounts of stored Information are operating* Such a program of research has 
the potential to develop the basis for more exacting analyses of problem 
solving tasks in terms of an eaq)licit theory of human memory, to elucidate 
the role of control and decision processes, and to qualify our understanding 
of memory mechanisms discovered through research on simple tasks* 
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statistical Appendix 

• ' ■ ' 

the coefficient vectors a « <a^,.»,,a^> define a classification of 
observations into distinct categories imder the laodel; forty-six such 
classification categories were observed in the e:q>erliDent (l.e.> there were 
46 distinct ^ vectors). The notation below ^11 be used in what^ follows • 
The index ?'l" refers to the i'**' subject and "j" -to the classification 
category! 



of classification categories 
the model 




8 


numbe^Tofsub^^t 




ninnber of observations 




RT sample mean 




RT grand sample mean 


^\ 


RT sample variance 




sample variance of 



(see methods in, Kendall & Stuart, 1969) 



Parameter estimation 

The approach taken to parameter estimation was to choose a loss 
ftmction conceived of as a function of tl^ parameters given the data, and to 
find parameter values that minimised this functloij* Since function minima 
were fotmd using a numerical grid search method, computationally efficient 
qiiadratid (least squares) loss functions were chosen* Parameters were 
estimated for each subject individually* 
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The actu^ estimation preceded in tvo steps « First values of e., 
and r^^ were determiTOd using the loss function. 



I-S, ( Si, q;, r^) 



i.i Si 



Second the parameter values, , » ,» were treated as constant and j 
estimated with the loss function, 

ii> ^0 - 2L -zr — ' 

An alternative procedure would have been to simultaneoxisly estimate all 
parameters using a combined loss ftmction of the form. 



/ LS»wLS3^ + (1-w)LS2^, 0<w<1 



However, it was observed that the RT means showed a clearer pattern than the 
RT variances, so that estimates of the mean RT parameters "uncon tami na ted " 
by possible failures of the model for^l^ variances were considered 
appropriate • 

Parameter estimates for individual subjects are listed in Table 3. 
The numerical method used to estimate variance parameters excluded negative 
variances with one result that some parameters were estimated to be near 
zero (the loss ftmction, LS^ , would have been reduced had negative values^ 
been accepted for these parameters)* An inherent problem in the analysis of 
RT variances is that for classification categories with small sample sizes 
the variability of the san?)le variance, Sj^^ , is large relative to that for 
the sample mean, H^^ : consequently parameter estimates will also have large 
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variability. Note ^that variance parameter estimates averaged across 
subject;'s are more readily interpre table as variability is reduced through 
averaging. 



Goodness of fit measures 



Consider the statistic defined for /the i^ subject and j*^ category 



by, , • ' 



which for suitable models may be assumed to be approximately distributed as 

y 

Student's t under the hypothesis that the theoretical mean» a-.'^i » is the 
true mean of the ij**^ RT distribution. One method of evaluating the fit of 
the model to mean RT's is to construct the smallest possible tmlform 
simultaneous confidence region containing all the t;^ *s and- to note the 
probability of the* complement of the region. This probability is the 
minimum value of dL (the probability of a type I error) for which the 
hypothesis that the model is true can be rejected; small values' indicate 
that the model is probably not a full account of the mean RT data. If the 
"distribution of t,^^ is approximated by N(OJ) instead of by Student's t, a 
conservative bias is introduced in the sense .that the value of <c Is 
necessarily reduced. Sinc^e . the normal appro:timation simplifies the 
calculation of a simultaneous confidence region this assumption is adopted. 
For the i**^ subjj&ot define, 

t* « max I t.. I 
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If the '8 were independent then for any positive number, c, 

Pr{ tf > c] - 1 - Pr(t* < c]. 

- 1 - TT Pr{-c < t^. < c] 

« 1 - [ Pr { -c < z < cj ^J'^ ' 

where Z'^COJI). This is the probability that for a fixed i all the t;,^ *s 
are contained within a iiniformy symmetric confidence band of width 2c* But 
for each i the t^^^ 's are correlated through the estimation procedure » and 
with enough parameters it may, be -posaible to obtain^ all the t;j • 0, 
rendering the preceding probability statements meaningless* Accordingly 
soma conservative adjustment should be made taking into account at least the 
number of free parameters » p. The choice for the present analysis was to 
take^ 

g*- 1- [Pr{-c<z<c}J 

in place of Pr{tf > c] above. If tf» t is, observed then g* is a 
statistical measure of the fit of the model for the i^ subject. Similarly 
for a saiiq>le of subjects define, 

t* • max I t* I - max | t- | 
then, 

S Cn-p) 



g* - 1- [Pr{-c<2<c]] 



is a goodness of fit measure for the sample as a whole* This procedure is a 
type of multiple modulus test (Miller, 1966) referred to here as a *Wximum 



modulus t test" with (n-p) or s(n-p) "degrees of freedom", taking some 
licence with terminology* « ! 

A related procedure can be followed in evaluating theoretical versus 
observed RT variances. The statistics defined by, 



t 



can be treated in the same manner as the tj^^ 's above although t^^^ cannot be 
regarded as having Student's t distribution and g* in this case ought to be. 
taken as a transformation of the t'^^ 's reflecting goodness of fit rather 
than aa-an approximation to a true probability. 

To obtain a firmer statement about goodness of fit a second measure 
was sought. Although thi model under consideration is not linear, the total 
sxai of squares can be partitioned in such a way as to yield statistics 
reflecting the goddness of fit of the model to RT means in a way similar to 
the percentage of between variance accounted for and the sample correlation 
coefficient in linear regression. Define 'for any set of theoretical means, 
(fl^j , for the i*^ subject. 



MV.^* « SS(between) - £ N., ( f.. - M-. T 

- 2 I i f,i - M,.) ( f.. ~ M,,) I 



4*1 



{ MV. , 0 J 



PBVj^ « 100 

SS (b'et^een) 
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PTVj^ • too I ^ 

SS (total) 



If the f^^ *8 were determined* under a linear regression model then, 

H7-^ « HV;, « SS (linear regression) 

The' results of the maximum modulus t» PBV and PTV analyses for RT 
means are presented in Table 4 ."Table 4a gives these statistics for the 
classification categories determined by the a vectors of the model; 
Table 4b represents the same analysis applied to the classification of 
Table 1 (problem type X position of false link) • From Table 4a it is clear 
that tl^e mode^ accounts for a fair proportion of the variance (average PBV 
and average PTV is 35.2), yet only two subjects have g* > .10 which 
^sonable" criterion for a good fit. Additional information about 
the maximum* modulus t test is given by the number o^ points falling* outside 
the .90 confidence region; g* > .10 if and only if this ntxmber is zero. It 
should be noted that points which lie outside the confidence region are not 
necessarily those which the model fails to account for since when parameters 
are ~ estimated sMultaneously for all points an "exceptional" point can 
adversely influence the prediction for other "normal" points. , For the groitp 
of six subj^<^t8yXhe maximum modulus t test Indicates that the model is true 
can be rejected for di » .004. It should be noted that one bad^ data point 
for a single subject can be sufficient to reject the model for the group 

using the maximum modulxis t test; the proportion of subjects for which the 

'I • 

model is not rejected is perhaps a more appropriateV group statistic. In 
view of the all too common practice in the literature of presenting 
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statistics for averaged group data it is difficult to make a firm statement 
on this point based on the results of other comparable analyses. 

The analysis presented in Table 4b indicates a slightly better fit 
although it is derived from a less strict interpretation of the model. Some 
impsjovement is expected since more extensive averaging may cancel out 
effects not accounted for by the< model and estimated error variance is 
increased slightly as observations with different means are pooled. 
However, this second classification does correspond to an intuitively 
natural division of the data, r 

Table 5 presents an evaluation of the model's success in accounting 
for RT variances. As remarked above the variance of S^- is large for small 
sample sizes: for the experimental data this renders the maximum modulus t 
test uninteresting because for individiial subjects, the T^^ 's are too large 
to reject any set of ballpark estimates for the variances. Variance 
predictions under the model were compared to the the hypothesis that all the 
s are the same, using the proportion of points better accounted for by 
the model (absolute differences between predicted and observed were 
compared). Referring to. Table 5, the model succeeds about as well as the 
"same" hypothesis for four subjects and does worse for the remaining two 
subjects' data. This is not strong support for the model applied to RT 
variances but may be interpreted to 'mean that, compared to the "same" 
hypothesis, attempting to infer stage variances did not cost much in the way 
of goodness of fit, while at the same time the model's predictions cannot be 
rejected given the variability of the S^j estimates. 
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